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~ The MAILING DATE of this communication appears on the cover sheet with the correspondence address- 

All claims being allowable, PROSECUTION ON THE MERITS IS (OR REMAINS) CLOSED in this application. If not included 
herewith (or previously mailed), a Notice of Allowance (PTOL-85) or other appropriate communication will be mailed in due course. THIS 
NOTICE OF ALLOWABILITY IS NOT A GRANT OF PATENT RIGHTS. This application is subject to withdrawal from issue at the initiative 
of the Office or upon petition by the applicant. See 37 CFR 1.313 and MPEP 1308. 

1 . |EI This communication is responsive to an amendment filed on 8/19/2008. 
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a) ^All b)DSome* c) □ None of the: 

1. ^ Certified copies of the priority documents have been received. 

2. □ Certified copies of the priority documents have been received in Application No. . 

3. □ Copies of the certified copies of the priority documents have been received in this national stage application from the 

International Bureau (PCT Rule 17.2(a)). 
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Applicant has THREE MONTHS FROM THE "MAILING DATE" of this communication to file a reply complying with the requirements 
noted below. Failure to timely comply will result in ABANDONMENT of this application. 
THIS THREE-MONTH PERIOD IS NOT EXTENDABLE. 

4. □ A SUBSTITUTE OATH OR DECLARATION must be submitted. Note the attached EXAMINER'S AMENDMENT or NOTICE OF 

INFORMAL PATENT APPLICATION (PTO-152) which gives reason(s) why the oath or declaration is deficient. 

5. □ CORRECTED DRAWINGS ( as "replacement sheets") must be submitted. 

(a) □ including changes required by the Notice of Draftsperson's Patent Drawing Review ( PTO-948) attached 

1 ) □ hereto or 2) □ to Paper No./Mail Date . 

(b) □ including changes required by the attached Examiner's Amendment / Comment or in the Office action of 

Paper No./Mail Date . 
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EXAMINER'S AMENDMENT 

1. An examiner's amendment to the record appears below. Should the changes 
and/or additions be unacceptable to applicant, an amendment may be filed as provided 
by 37 CFR 1.312. To ensure consideration of such an amendment, it MUST be 
submitted no later than the payment of the issue fee. 

Authorization for this examiner's amendment was given in a telephone interview 
with Greg Hsu on Thursday, October 2, 2008 

The Examiner has made the following changes to the claims. Additions to the 
claims are reflected by underline ( example ) and deletions are reflected by strikethrough 

Claim 1 

An interconnecting neural network system comprising: 

a neural network unit that includes a plurality of neurons, each of the neurons 
embodying a weight holding unit, a pointer unit, a duration variable holding unit and an 
activation time holding unit, each of the neurons outputting an excitation strength 
according to a similarity between input vectors and centroid vectors based on a kernel 
function; and 

a network control unit that constructs an artificial neural network structure by 
interconnecting the neurons relating to each other among the neurons in the neural 

, | Deleted: 

network unit via a /especl;ve weights, 
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wherein each of the neurons in the neural network unit outputs an excitation 
strength according to the similarity between the input vectors and the centroid vectors 
based on the kernel function when each neuron is excited by the input vector applied 
from an outside, and outputs a pseudo excitation strength obtained based on an Jhe 
sum of excitation strength outputs from the other neurons when each neuron is excited 
in a chain reaction to excitation of the other neuron connected to each neuron, 

wherein each of the neurons in the neural network unit has a plurality of 
modalities different from one another, the plurality of modalities of the neurons including 
auditory modality and visual modality so that a plurality of different input vectors of 
auditory modality and visual modality are handled simultaneously and independently by 
the neurons to perform auditory and visual recognition concurrently . 

Claim 8 

A computer-i mplemented. ..method of constructing an interconnecting neural 
network structure, the method comprising the steps of: 

preparing an artificial neural network structure including a plurality of neurons, 
each of the neurons embodying a weight holding unit, a pointer unit, a duration variable 
holding unit and an activation time holding unit, each of the neurons outputting an 
excitation strength according to a similarity between input vectors and centroid vectors 
based on a kernel function, the neurons relating to each other interconnected in the 
artificial neural network structure via a .respective weights^ and 



Application/Control Number: 10/806,090 Page 4 

Art Unit: 2129 

training the weight that connects the neurons to each other, based on the 
excitation strength of each neuron, 

wherein each of the neurons in the artificial neural network structure has a 
plurality of modalities different from one another, the plurality of modalities of the 
neurons including auditory modality and visual modality so that a plurality of different 
input vectors of auditory modality and visual modality are handled simultaneously and 
independently by the neurons to perform auditory and visual recognition concurrently . 

Claim 13 

A computer-implemented method of constructing a self-organizing neural 
network structure including a plurality of neurons, each of the neurons embodying a 
weight holding unit, a pointer unit, a dura tion variable holdin g unit and an activation time 
holding unit, each of the neurons outputting an excitation strength according to a 
similarity between input vectors and centroid vectors based on a kernel function, the 
neurons relating to each other being autonomously connected in the self-organizing 
neural network structure based on the input vector, the method comprising: 

a first step of adding a neuron, which has input vectors as centroid vectors for a 
kernel function, into the self-organizing neural network structure as a new neuron based 
on input vectors that is input first from an outside; and 

a second step of repeating the following processings (a) to (c), each of the 
processings being based on input vectors that is an n 1h input vector from the outside, 
where n is an integer equal to or greater than 2: 
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(a) the processing of calculating excitation strengths of all the neurons in the 
self-organizing neural network structure based on the n th input vector input from the 
outside; 

(b) the processing of adding a neuron, which has the n ,h input vector as a 
centroid vector for a kernel function, into the self-organizing neural network structure as 
a new neuron in case that it is determined by the processing (a) that there is no neuron 
excited such that the excitation strength thereof exceeds a predetermined threshold, 
among one or a plurality of neurons in the self-organizing neural network structure; and 

(c) the processing of performing both of or one of formation of a weight that 
connects the neurons, and training of the formed weight based on the excitation 
strengths of the neurons in the self-organizing neural network structure; 

wherein each of the neurons in the self-organizing neural network structure has 
a plurality of modalities different from one another, the plurality of modalities of the 
neurons including auditory modality and visual modality so that a plurality of different 
input vectors of auditory modality and visual modality are handled simultaneously and 
independently by the neurons to perform auditory and visual recognition concurrently . 

Claim 20 

An interconnecting neural network system comprising: 
a plurality of intermediate layer neurons, each of the intermediate layer neurons 
embodying a weight holding unit, a pointer unit, a duration variable holding unit arid an 
activation time holding unit, each of the intermediate layer neurons outputting an 
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excitation strength according to a similarity between input vectors and centroid vectors 
based on a kernel function, and each of the intermediate layer neurons using centroid 
data in a matrix form in light of time series changes as the centroid vector; and 

an output layer neuron connected to each of the intermediate layer neurons and 
outputting a change in the excitation strength output from each intermediate layer 
neuron at time series, 

wherein each of the neurons has a plurality of modalities different from one 
another, the plurality of modalities of the neurons including auditory modality and visual 
modality so that a plurality of different input vectors of auditory modality and visual 
modality are handled simultaneously and independently by the neurons to perform 
auditory and visual recognition concurrently . 

Reasons for Allowance 

2. The following is an examiner's statement of reasons for allowance: claims 1-5 
and 18-21 are considered allowable since when reading the claims in light of the 
specification, as per MPEP §2111.01 or Toro Co. v. White Consolidated Industries Inc., 
199 F.3d 1295, 1301, 53 USPQ2d 1065, 1069 (Fed. Cir. 1999), none of the references 
of record alone or in combination disclose or suggest the combination of limitations 
specified in the independent claims, specifically a plurality of neurons, each of the 
neurons embodying a weight holding unit, a pointer unit a duration variable holding unit 
and an activation time holding unit (as defined at e.g. pg. 12, L25-28 and Figure 3 of the 
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specification of the instant application) described in claims 1, 8, 13 and 20 of the instant 
application. 

Gurney ("An Introduction to Neural Networks") discloses an explanation of Neural 
Networks using Radial Basis Function neurons. 

Hatayama et al. (US Patent #6,219,657) discloses an emotion creating device 
that uses a neural network. A user inputs information representing conditions of a user 
and outputs the user's emotional state. 

However, none of the references alone or in combination disclose the 
combination of limitations of claims 1, 8, 13 and 20, especially a plurality of neurons, 
each of the neurons embodying a weight holding unit, a pointer unit a duration variable 
holding unit and an activation time holding unit. The references fail to teach the 
invention without unduly motivating one of ordinary skill in the art to exert substantial 
experimental effort to integrate such combination concepts. Further to the 
Examiner's Amendment cited above, the Applicant's Remarks in the response dated 
8/18/2008 are acknowledged and accepted. 

Conclusion 

3. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Engel et al. US Patent #5,761 ,383 

Hoya US PGPUB #2003/01 15165 

Kramer et al. US Patent #5,335,291 



Application/Control Number: 10/806,090 
Art Unit: 2129 

4. Claims 1-5 and 18-21 are allowed. 
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If you need to send an Official facsimile transmission, please send it to (571 ) 
273-8300. 

If attempts to reach the examiner are unsuccessful the Examiner's Supervisor, 
David Vincent, may be reached at (571) 272-3080. 
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Service Window Randolph Building 401 Dulany Street Alexandria, VA 22313), located 
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Patent Examiner 
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